DOCOHENT RESOHE 



ED 142 439 



SE 022 908 



AUTHOR 
TITLE 

INSTITUTION 
SPONS AGENCY 
PUB DATE 

note' 



EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



Dougherty, Knowles; Herbert, Martin 

Evaluation Report 2-A-1. Final Summary Report, Year 

2. 

Central Midwestern Regional Educational Lab., Inc., 
St. Louis, Ho. 

National Inst, of Education (DHEW) , Washington, 
D.C. ' 
1 Nov 75 

35p.; Extended Pilot Trial of the Comprehensive 
School Mathematics Program; For related documents, 
see SE 022 895, 909-916 and ED 101 993-ED 102 007 

MF-$0.83 HC-$2.06 Plus Postage. 

♦Curriculum; Elementary Grades; *Elementary School 
Mathematics ; *Evaluation ; Instruction; *Instructional 
Materials; Mathematics; Mathematics Education ; 
♦Program Evaluation 

♦Comprehensive School Mathematics Program 



ABSTRACT 

The Comprehensive School Mathematics Program (CSMP) 
is a program of CEMREL, Inc., one of the national educational 
laboratories, and is funded by the National Institute of Education. 
Its major purpose is the development of curriculum materials in 
mathematics for grades K-6. Beginning in September, 1973, CSMP began 
an extended pilot trial of its Elementary Program. This report is an 
attempt to summarize the information collected during the second 
year, 1974-75. During the second year, 30 school districts were 
involved in the use of some combination of kindergarten, first, and 
second grade materials. The report includes data on the school sites, 
review of formal written reports, and a review of evaluation 
questions. (RH) 
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Description of Evaluation Report Series 



The Comprehensive School Mathematics Program (CSMP) is a program of CEMREL, 
Inc., one of the national educational laboratories, and is funded by the National 
Institute %f Education, Its major purpose is the development of curriculum 
materials for grades K-6. 

Beginning in September, 1973, CSMP began an extended pilot trial of its 
Elementary Program. The pilot trial is longitudinal in nature; students who 
begarn usj.ng CSMP materials in kindergarten or first grade in 1973-74, were able 
to use then: in first and second grades r»^.spectively in 1974-75, and will be able 
to use them in second and third grades i.: 1975-76. Hence the adjective "extended". 

The evaluation of the program in this extended pilot trial is intended to be 
reasonably comprehensive and to supply information desired by a wide variety of 
audiences. For that reason the reports in this series are reasonably non-technical 
and do not attempt to widely explore some of the related research issues. The list 
of reports from the. first two years of the extended pilot trial is given on the 
next page. The most comprehensive*, of these are the following: 



and 



1-A-l: 
•l-A-3: 
.>A-1: 



Overview, Design and Instrumentation 
Final Suimnary Report, Year 1 
Final Summary Report, Year 2 



The first of these v;ill be j^articularly useful to the reader in providing a 
description of the rrogiim, the philosophy and goals of the evaluation and 
the relationship ot individual reports to the evaluation effort as a whole. 
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Key to Indexing 
Observations of CSMP First Grade Classes 

"2" refers simply to the number within a given year and type of data 

refers to the type of data being reported 

A: Overview, summary and theoretical reports 
B: Student outcomes 
C: Non-test data 

refers to the year of the Pilot Study according to the following: 
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Introduction 

In the fall of 1973, the Comprehensive School Mathematics Program (CSMP) 
began a longitudinal pilot study of its Elementary Program. Over 100 teachers 
began using the program, either in first grade or kindergarten. During the 
1974-75 school year, the second year of this pilot study, most of these classes 
continued into second grade and first grade respectively and many new classes 
began ''using CSMP materials. 

During the first two years of this Pilot Trial a considerable amount of 
information was collected about the program, how it was being used and with 
what results. The list of Evaluation Reports on page iv will give the reader 
some idea of the variety of evaluation tasks carried out. With the exception 
of the first and third reports, 1-A-l and l-A-3, each report deals with a spe- 
cific issue or set of data. Report 1-A-l is intended to be an introduction to 
the series and describes the program in terms of its content and materials and 
its history, the general and specific objectives of this suiranative evaluation, 
and the setting and design of the Pilot Trial. Report l-A-3 is a summary of 
ail the reports written on the studies conducted during the first year of the 
trial, 1973-74. 

The present report is an attempt to summarize, in a ;.easonably non-technical 
way, the information collected during the second year. While it is not possible, 
nor necessarily desirable, to suppress the opinions and prejudices of the evalu- 
ator, one hopes the reader can easily separate the presentation of data from the 
authors' interpretation of those data. In any case, if the reader wishes more 
information about certain of the data reported here or would like to see the 
actual tests or questionnaires or instruments used, he or she may consult the 
appropriate report in this Evaluation Report Series. 
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Setting: The Second Year 

In the first year of the CSMP Extended Pilot Trial, 29 school districts 
decided to try out the CSMP kindergarten and/or first grade materials. For 
the second year 24 of the 29 districts continued in the EPT in various ways 
and six new districts were added.* For the resultant 30 districts the continu- 
ation involved the use of some combination of kindergarten, first and second 
grade materials. 

As was the case in the first year, no conditions were placed on the 
number or location of pilot classes so that participating school systems were 
free to use the materials in as limited or extensive a way as they wanted and 
with whatever kinds of classes or teachers they wanted. They were required to 
pay the cost of producing the necessary instructional materials, to provide or 
allow for the collection of relevant evaluation data and to provide a coordina- 
tor for the program in their district. This coordinator was responsible for 
overseeing the implementation of the program including the training of pilot 
teachers (except as noted below) and. was generally the liaison person between 
CSMP and the local district. The coordinator was also required to attend a 
one-week training workshop in the use of the CSMP curriculum. 

"Local" and "Outer Ring" Sites 

The same eight districts as last year were located in the St. Louis ^rea 
and were designated "local" as opposed to "outer ring," For them the same 
additional conditions were imposed as last year. First, for each second grade 
pilot class a suitable comparison class was to be establirhed and accessible 
for the collection of various data. Second, it was understood that a considerable 
amount of "evaluating" would be done in these local classes, including interviews, 
observations, and group and individual testing. Third, all local pilot teachers 
of "lead" classes (for 1974-75, second grade classes) were trained at one of the 
series of one-week summer training workshops conducted by CSMP. Any further 
periodic training sessions to be held during the school year were the responsibility 
of the local coordinators. 

Thus local classes tried out the materials in fairly wel] -controlled 
circumstances with standardized training, comparison classes, and considerable 
observation, while classes at distant sites in the "outer ring" implemented 
the program in a much less restricted way. 

Below is a double table listing the 8 local and 22 outer ring sites along 
with the nu'nbers of kindergartens, first and second grades involved in the EPT 
in each site. 



*The five districts that dropped out of the EPT had had 10 kindergartens 
and four first grades. ITie six new districts had 13 kindergartens, seven 
first grades and two second grades. 
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Table 1 





Number 


of Claodes 




^'Local*' School Districts 


Kindergarten 


FirsL ornJe 


Second Grade 


Affton, MO 


1 




0 


0 


Ferguson-Florissant . MO 


6 




7 


2 


Francis Howell, MO 


0 




5 


5 


Herrin, XL 


1 




1 


0 


Ladue, MO 


6 




6 


0 


Normandy, MO 


6 




12 


4 


Saint Louis, MO 


13 




11 


3 


Sesser, IL 


3 




3 


3 






Number 


of Classes 




"Outer Ring" School Districts 


Kindergarten 


First Grade 


Second Grade 


Austin, TX 


3 




3 


4 


Autrain Township, MI 


1 




0 


0 


Rp^h^phr>m PA 


15 




8 


2 


Clarksville Academy, TN 


0 




1 


0 


Clarksville/Montgomery Co., TN 


5 




8 


1 


Detroit, MI: Region 4 


10 




12 


0 


Region 5 


• 9 




11 


6 


Elizabethtown, KY 


1 




2 


1 


Ennis, TX 


8 




0 


0 


Fort Campbell, KY 


1 




2 


1 


Gulliver Academy, FL 


1 




1 


0 


Houston Co. , TN 


3 




2 


0 


Lincoln Co. , TN 


2 




2 


0 


Marquette Diocese, MI 


■« 






1 


Nashville, TN 


1 




1 


1 


New Hartfi^rd, NY 


2 




2 


1 


North Allegheny, PA 


18 




0 


0 


Philadelphia, PA 


0 




8 


4 


Polk Co., GA* 


7 




29 


25 


Portland, ME 


15 




14 


0 


Shippensburg, PA 


I 




1 


1 


Tenn. Tech. Univ. Exp. Sch., TN 


1 




1 


1 


Walker Co., GA 


0 




3 


2 


*Polk Co., GA, also had 16 third 


grades (using 


secon 


d grade materials) . 
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Coordinators and Teacher Training 

As was the case in the ':irst y v, the people who became CSMP coordinators 
could be put into one of four (;.-^L ;3or Les according to their position within the 
school district. Some were re. g 0 : teachers who filled the double role of 
pilot teacher and coordinator. 'ior.)-:» coordinators were elementary school principals. 
Sor?.e coordinators were prima- ily collage based personnel who worked with a school 
district on a somewhat ad hoc basis in implementing this pilot trial. The largest 
group of coordinators were district supervisory officials, usually with a title 
such as curriculum coordinator or curriculum consultant. Thus there was consider- 
able variation in the role the coordinator ordinarily played in the school system, 
in the authority over pilot teachers which that role provided and in the mathe-- 
matics background of coordinators. 

As noted previously, all coordinators, as well as all second grade pilot 
teachers in the St. Louis area^ attended a one-week training workshop in the 
summer. The workshop time was about equally divided between the development 
of mathematical content and the methods used to teach the content, including 
the use of various instructional materials. The presentation was in lecture 
format for the most part with some questlon-and-answer sessions. Experienced 
teachers were available to help answer questions and specially prepared films 
were also used. A more complete dec ^ription of the CSMP v»orkshops (for kinder- 
garten and first grade teachers) is given in another evaluation Report.* 

Upon their return to their local district, outer ring coordinators were 
responsible for the training of their own pilot teachers. In addition, all 
the coordinators served as liaison between CSMP and the local school system 
and collected any required evaluation data from the local system. 

T he "Local'* Setting and the Emphasis of the Evaluation Effor t 

There were many new kindergarten, first, and second grade classes started 
nationwide in 1974-75, but the major evaluation emphasis was on the local second 
grade classes of students who would be completing their second year of the CSMP 
curriculum. After the first year of the Extended Pilot Study, school districts 
made the decision of whether or not to continue using CSMP in second grade with 
the original class of first graders. Of the 8 original local districts, one 
continued with kindergarten classes only, two districts continued with 
kindergarten and first grades only, and one had adopted the program district- 
wide making it difficult to locate comparison classes. 

With the exception of the teacher questionnaire, the data summarized 
here comes exclusively from these local districts: the questionnaire being 
sent to teachers both .within and outside the local area. With the exception 
of the two sets of data collected on first grade students and teachers (from 
the same districts described above), the data reported herewith is on the 
second grade teachers and students described in Table 2 below. 



*Evaluation Report l-C-l- Teacher Training Report 
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Table 2 



Description of Local 
Second Grade CSMP and Non-CSMP Classes, 1974-75 









Predominant 




Es 


timated 


Socio- 








Number of 


Mean Number 


Racial Makeup 


Economic Status 




















0) 

i-i 






District 


Pairs of 


of Students 
















Type of 


Number 


2nd Grade 
Classes 


per CSMP 
Class (non- 
CSMP Class)* 


1) 


'White 


ixed 




Lo'.-'-Midd; 


Lddle 


Lgh Mid< 


00 


Community 








rH 




0 










1 


2 


21(20) 




/ 








/ 






Suburban 


2 


5 


22(24/ 




/ 






/ 


/ 






Rural/Suburban 


3 


3 


25(25) 










/ 




/ 




Suburban 


1/ 


3 


27(28) 


/ 






/ 


/ 








Urban 



*Based on numbers present when covariate test was administered. 
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Review of Formal Written Reports 

Second Grade Test Dat a* 

Thirteen second grade classes in four districts in the local area studied 
the CSMT curriculum during the 1974-75 school year. A wide range of abilities 
and socio-economic status was represented by these classes (see Tables 2 and 3), 
For each CSMP class there was a comparison class, in the same school where 
possible and in an adjacent school where the CSMP class was the only second grade ! 
class. These classes were usually the same groups of students who were tested 
the year before in first grade in the CSMP - Non-CSMP comparisons described in a 
previous report. ** 

The design was essentially the same as for the previous year. The comparison 
classes established the previous year were, with two exceptions, continued 
virtually intact as second grade comparison classes. In those two cases, new 
comparison classes were established. The Kuhlmann-Anderson Test, a test of 
mental ability, was administered to all classes in the fall, and these scores 
were used as covariates for adjusting scores on the basis of differing entering 
ability. This test will be referred to subsequently as the "pi^etest**, though 
not in the sense of a pre-post test designed to measure change. As will be 
seen later such adjustments were very small because of the excellent matches 
between the CSMP classes and their paired comparison classes. Table 3 summarizes 
the situation below. 



Table 3 



Local Second Grade Classes 
in Extended Pilot Study, 1974--75 







CSMP Claaa 


Conpariaon Cl^iaa 




District 


Claaa Pair 


Maan Preteat 


Number of 


Hc«n Pratnac 


Nuaber of 


Claaa Palra Located 


Nuad}er 


Number 


Score 


Studenta 


Raw Score 


Studenta " 


In Same School? 


1 


1 


51.5 


17 


1^9.6 


17 


Yea 




2 


52.4 


24 


51.7 


23 


Yea 




3* 


62.3 


25 


59.6 


25 


Yes 


2 


A 


42.5 


21 


49.7 


19 


Y«fa 




5* 


52.4 


23 


49.1 


20 


Yes 




6 


56.5 


22 


55.2 


25 


Yes 




7 


58.9 


15 


58.2 


28 


No 




8 


54.2 


25 


53.5 


24 


Yes 


3 


9 


54.3 


28 


49.4 


25 


Yes 




10 


52.5 


23 


49.7 


27 


Yea 




11 


36.1 


27 


31.9 


28 


No 


k 


12 


42.3 


27 


45.0 


26 


No 




13 


45.0 


25 


50.4 


30 


No 




Maaa 


50.8 


23.2 


50.2 


24.4 





<*Claaa pair 3 and claaa pair 4 vara locaCad In the aama achool. So vara claaa palra 5 and 6. 



*For a complete report on this data, see 2-B-l, Second Gra de Test Data 
**Evaluation Report l-A-3: Final Summary Report Yea r 1 
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Two kinds of tests, a standardized test and a series of comparison tests, 
were administered to CSMP and Non-CSMP classes according to a testing plan 
which ensured that representative pairs of classes would take each test. In 
the case of the standardized test all 13 pairs of classes were used. The 
standardized test used was the Mathematics Test, with two subtests, of the 
Comprehensive Test of Basic Skills. Four Comparison Tests with varying numbers of 
subtests were developed: two tests were group-administered to the whole class and two 
administered to about seven representative students of each selected class. 
These comparison tasks were essentially problems posed in situations which 
were novel to both CSMP and Non-CSMP students. They were intended to be 
situations in which the techniques and ways of thinking about mathematics 
which are stressed in the CSMP curriculum might he used with more success 
than would normally be the case. 

Table 4 summarizes the results of thes^ri tests- Mean scores across classes 
are given for the CSMP and for the Non-CSMP classes, and those tests on which 
the differences were significant at the .05 level are indicated. A two-way 
Analysis of Covariance was used, with classes as the unit of anlaysis, and 
the resulting F-test had between 1 and 4, and 1 and 11 degrees of freedom. 



Table 4 

Summary Data for Tests 
Administered to CSMP and Non-CSMP Classes 





Number of Pairs 


Mean for 


Mean for 


Significant 


Test 


of Classes 


CSMP Classes 


Non-CSMP Classes 


St .05 level 


Comparison Test I 










•) Word Problems 




4.79 


4.06 




b) Number Puzzles 




2.66 


2.51 




c) Estimation: Calculation 


6 


5.18 


5.03 


/ 


d) £/;timatlon: Largest Number 




3.26 


4.44 


e) Showing Fractions 




1.15 


0.92 




Total 




19.06 


17.01 




Comparison Test II 








/ 


a) Equations: Construction 




2.72 


2.03 


b) Equations: Fluency 


7 


13.63 


11.19 


/ 


c) Combinatorics 




16.94 


17.08 




d) Regrouping 




6.34 


6.55 


/ 


Total 




39.62 


38.86 


Cojsparlson Test III 










a) Class^.f icatlon 


12 


17.98 


17.23 




b) Binary Relations 




8.05 


8.12 




Total 




26.03 


25.36 




Conporlson le^c IV 








/ 


m) Number Patterns 


11 


10.11 


9.28 


b) Functions 




6. 48 


5.75 




c) Probability 




4.59 


3.99 


/ 


Total 




21.18 


19.01 


/ 


Coaprebenslve Test of 










Basic Skills 










a) Computation 


13 


21.63 


20.28 


✓ 


b) Concepts and Applications 




18.96 


18.12 


/ 


Total 




40.61 


38.47 


/ 
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All differences which were significant were in favor of CSMP, These included 
both the Computation and Concepts parts of the standardized test and several of 
the various Comparison Tests, With the exception of Comparison Test IVc) , Proba- 
bility, the four subtests on which CSMP classes did significantly better than 
Non-CSMP classes were what might be called "directly numerical" situations; 
numbers (or numerals) themselves were the stimuli. Other subtests in which the 
given stimuli were situations in which numbers (or more widely, mathematical 
thinking) were to be applied (la, le, lie, Ild, Ilia, Illb) did not produce 
significant differences. 

These results are very encouraging. CSMP students did better on both parts 
of the standardized test and on several novel tasks of a numerical nature. It 
IS also important to note that the results were consistent across school districts 
and, perhaps more importantly, across ability levels. It is not the case that 
differences were due to gains by high ability students; low ability students 
also did better. In addition, students who transferred into CSMP at the beginning 
of second grade appeared to do as well as students who were in CSMP from first 
grade except on Comparison Test I where they had slightly lower scores. 

In the authors' opinion there are two explanations worth considering other 
than the obvious one that the CSMP curriculum is responsible for these differences. 
The first explanation is that teachers were not randomly assigned to CSMP and Non-CSMP 
classes (indeed this is virtually impossible to accomplish in the pilot stages 
of any program implementation). It is therefore possible that CSMP teachers may 
be, as a group, more capable of producing student achievement. Tltiis is a plausible 
explanation and cannot be entirely refuted. However, in discussions with principals 
who had both a CSMP and a Non-CSMP teacher in their school, there was no support for 
such differences; indeed in one case the same teacher taught both classes and in 
another the principal thought the Non-CSMP teacher was a better teacher. It is also 
true that this selection factor would be stronger in the first year of this study 
when the very first teachers to use CSMP are chosen. The next year, the second 
grade teachers more or less inherit the program and the classes. 

The other explanation may be that CSMP classes spend more time in math 
instruction than do Non-CSMP classes. Based on interview data*, the mean number 
of minutes per day on math instruction in the 13 CSMP classes was 50, which is 
slightly higher than usual. However, there was no relationship between number 
of minutes and achievement either with or without adjusting for differences in 
entering ability. This information is not available for Non-CSMP classes though 
it is known that, in at least some classes, school scheduling dictated that the 
same time be spent for all math instructions. 

Thus, while these two alternatives cannot be ruled out, nhere are good 
reasons for discounting them as explanations for higher achievement by CSMP 
students. The so-called "novelty effect" is also discounted for two reasons. 
First, as described in the report dealing with student interviews**, students 
were usually not aware that their math program was particularly different from 
what other students used. Second, data collected over the past two years from 
first grade students indicates that, as judged by student performance, teachers 
do as well if not better in their second year of teaching CSMP as they did the 
first year (a fuller discussion is ..given on p. 9). 



^Evaluation Report 2-C-2: Teacher Interviews, Second Grade 
**Evaluation Report 2-B-3: Student Interviews 
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One is led to believe from what is presented above that CSMP students did 
significantly better than Non-CSMP students pn several measures and that this 
can be attributed to the CSMP program and not to other factors. The. reader 
should bear in mind that these results can not necessarily be generalized to 
school systems or classes or teachers beyond those participating in this 
study (ie. "local classes"). That is one of the penalties for non-random (in 
this case voluntary) sampling of school districts, and within districts, teachers. 

A series of tests dealing with the unique aspects of CSMP, content not 
already covered in one of the above tests, was also administered to CSMP classes. 
The results are more difficult to interpret bo. .-use of tlu lack of standards, 
either through what "usually happens" (contiol groups) or what "should happen" 
(a priori standards of achievement laid down by the program). Thus the interpre- 
tation of the data here is the authors' own subjective evaluation of what one 
should expect second grade CSMP students to be able to do. However, this evalu- 
ation is based on knowledge of curriculum, the contents of the various lessons 
and relative emphasis of topics; and on discussions with teachers, observations 
of classes and interviews with students. 

Rather than reviewing and commenting on the results of each subtest*, this, 
discussion will center on general topics. 

a) Multiplication. The level cf achievement with simple, whole-number, 
single-digit, multiplication is quite satisfactory, as indicated also 
by the standardized test data. It is also very satisfactory in two 
areas not typically given much consideration in second grade. A 
surprising number of students were able to double or triple relatively 
large numbers; nearly half the students could calculate "2x37". And 
in calculating fractional parts of a whole number, specifically 1/nth 
of a number, students also did very well; about half the students 
could calculate "jX48" and "ix20." 

b) Integers. Negative integers are not studied in other second grade 
curricula. Hence there is no basis of comparison and it may safely 
be said that whatever progress CSMP students make is a "gain" over 
Non-CSMP students. The level of success is judged to be adequate; 
60% of the students could select the one true number sentence of 
four very similar and possibly confusing sentences involving negative 
integers, though only about half the students could compute "4+7" 
and "5+4." However, the improvement from first grade is rather dis- 
appointing. There was only a moderate increase from last year in the 
percentage correct on those items repeated from last year's test. 

c) Arrow Diagrams. The method of looking at percent correct is not 
entirely satisfactory for this topic. Generally these percentages 
are satisfactory. On the one hand there are surprisingly high 
percentages of students getting difficult items correct; for 
example the "Deter' r i.ve Story*' and other items whose format was 
completely new to students. On the other hand there were many 
subtests on which up to 20% of the students had virtually no 
success. 

d) Minicomputer. Again there are a very wide range of scores; many 
could do virtually everything asked and many others could do 
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virtually nothing. Over 30% of the students made, at most one error 
on this rather comprehensive individually administered test and 
these students generally worked very rapidly. On the other hand 
when considering low ability students (the bottom 20% of the CSMP 
students as measured by the pretest), only two of these 11 students 
could consistently read and write numbers on the Minicomputer, let 
alone even set up the Minicomputer for subtraction, multiplication 
or taking one-half of a number. 

Students who transferred into a second grade CSMP class during the first two 
months of school do not, as a group, perform as well on these tests of CSMP con- 
tent as students who were in the program from the beginning, '/hus while new 
students do not appear to be penalized as far as the standard skills and concepts, 
they do not, during the course of the year, "catch up" with the other students in 
acquiring those concepts specific to CSMP. This is not a surprising result. 

In suiranary then, CSMP classes did better than Non-CSMP classes on both stan- 
dardized measures and some "content-free" comparison tasks, and this is true for 
both high and low ability students and for both original and transfer students. 
Although overall performance was generally adequate and at times praiseworthy 
for unique CSMP content, on those subtests involving arrow diagrams and the 
Minicomputer a considerable number of students had little or no success, and 
'students who transferred into CSMP classes near the beginning of the year were 
not able to catch up with their classmates on CSMP concepts. 

Readminist ration of First Grade Test Items* 

The major purpose of this study was to compare the performance of two groups 
of first grade CSMP students; the teachers of the first group were new to CSMP 
and the teachers of the second group were these same teachers one year later. A 
lower performance the second year would support the contention that results** 
obtained in the first year of the pilot study, 1973-74, were at least partly due 
to the enthusiasm of the teachers for a novel math program, A higher performance 
would support the view that teachers the second time around more than made up for 
any loss of enthusiasm because they were more familiar with the program. 

The classes used to investigate this issue were those of teachers 1-12 in 
the chart below, ie. , all local teachers who began using CSMP in 1973-74 and 
continued using it in 1974-75. 



*For a complete report on this data, see 2-B-2: Readmlnis tration of First 
Grade Content ~ ~ 

**Evaluation Report l-B-2: End-of-Year Test Data; CSMP First Grade Content 
i-T^?^" Evaluation Report l-B-3: End-of-Year Test Data: Standard First Grade Content 
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The classes of teachere 1* - 5* in the above chart provide information regard- 
two other issues. One addresses the possibility that CSMP teachers in the first 
year of the pilot study were systematically better than teachers who served as com- 
parison teachers. The performance of the students of teachers 1-5 in their first 
year of teaching CSMP, 1973-74, is compared with performance of the students of 
teachers 1* - 5* in their first year of teaching CSMP, 1974-75. 

The other issue is a partial replication of the CSMP - Non-CSMP comparisons 
made originally in 1973-74. To do this, the performance of the students of 
teachers 1* - 5* in 1973-74 (using a traditional math text) is compared with the 
performance of the students of these same teachers one year later (using the CSMP 
program) . 

In all these comparisons an assumption has been made that the general ability 
level of students within any of the participating schools has not changed from 
1973-74 to 1974-75. 

Items were selected from a battery of tests administered che previous year to 
CSMP and comparison classes. Sixteen subtests x^zere developed covering 11 content 
topics. This set of subtests was divided into three parts. Within each of the 
17 classes one part was administered to a random half of the students and the 
third part was administered to all students. The administration of the testing 
was carried out by two specially trained testers. 

A univariate analysis of variance was used on the class means of each sub- 
test. A comparison was made between student performance with inexperienced 
teachers and with the same teachers after one year's experience with CSMP and 
revealed the following: 



18 



12 



a) Classes with an experienced CSMP teacher scored significantly 
higher (p<.05) on Subtest 16: Larger Numbers. 

b) Classes with an experienced CSMP teacher scored higher (marginally 
significant with p<.06) on Subtest 11: Relations - +3 spiral. 

c) There were no differences approaching significance on the other 
14 subtests. 

A review was made of the 11 items answered correctly by at least 10 percent 
more students with experienced CSMP teachers than students with inexperienced 
CSMP teachers. These items generally dealt with one of three content areas: 

a) Addition of integers 

b) Estimation of sums of two two-digit numbers 

c) Arrow diagrams ^ 

One item was answered correctly by at least 10 percent more students with an 
inexperienced CSMP teacher than students with an experienced CSMP teacher, namely 
a subtraction problem, 3-0= . 

10-70 ^,""^P^^^s°" °f fi^e CSMP classes in 197A-75 versus five Non-CSMP classes in 
1973-74 with the same teachers revealed the CSMP classes scores significantly 
higher (p<.05) on Subtest 16: Larger Numbers. 

A comparison of five CSMP classes in 1973-74 (taught by an original group of 
CSr-IP teachers) with five CSMP classes in 1974-75 (taught by the five teachers who 
served as comparison teachers in 1973-74) indicated the comparison teachers did 
at least as well when teaching CSMP for the first time as did the original croup 
of teachers. © a f 

In conclusion, it appears that CSMP teachers do just as well the second time 
around and that any loss due to the novelty effect of the program is more than 
balanced by the gain in experience. This is in agreement with statements made 
by these same teachers in a series of interviews conducted at the end of the 
sclxpol year (see Evaluation Report 2-C-3) . In addition, the data tends to corrobo- 
rate the findings from the first year of the pilot study and to cast some doubt on 
the possibility that in some casas CSMP students did better because they had better 
teachers. While the results are not strong enough (small numbers of teachers, no 
covariate scores such as pretest ability scores) to support a definite conclusion 
they tend to give additional credence to the first year results. ' 

Student Interview s* 

During the second half of the 1974-75 school year a series of 4 interviews 
was conducted with 18 students in 9 second grade classes. The students were 
selected to provide a total group which was representative of the entire group 
of local second grade CSMP students. 



*See 2-B-3, Student Interviews , for a complete report. 
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The first two interviews were designed to survey students' knowledge and skills 
in CSMP mathematics. The third interview was a rather individualized follow-up and 
extension of Interviews 1 and 2 and included questions dealing with elementary con- 
cepts, repetition of certain kinds of items and a series of problem situations deal- 
ing with applications of certain CSMP concepts. The fourth interview was of a sum- 
mative and attitudinal nature. 

The complete set of interviews and student responses are given in the relevant 
Evaluation Report. The enormous amount of information from these interviews cannot 
readily be summarized and the reader is urged to read the actual interviews and 
form his -jwn opinion of what conclusions to draw from them. 

Teacher Questionnaire Data* 

Near the end of the 1974-75 school year a rather extensive questionnaire was 
sent to all local and outer ring teachers. Slightly different forms of the 
questionnaire were prepared, depending on whether the teacher was teaching the 
kindergarten, first or second grade portion of the program. Some of the kinder- 
garten and first grade teachers were using the program for the second year and, 
where feasible, the analyses of responses in this report were done separately 
for these ''experienced*' teachers. The questionnaire was sent to approximately 
360 teachers. About 50% of the questionnaires were returned. 

In this summary of the data collected, not every single item will be attend- 
ed to. What will be summarized are what the authors consider the key questions. 
Special emphasis will be placed on areas in which significant dissatisfaction 
was expressed. 

Students' attitudes towards CSMP were very favorable. The clearest, most 
unequivocal responses were given in answer to questions concerning student 
attitude. On two items comparing students' attitudes towards CSMP with their 
attitudes towards a traditional math program (a free response item and a 
multiple fhoice response) there was very strong agreement that students enjoyed 
CSMP and wfere enthused about it and much preferred it to a traditional program. 
In the overall evaluation of CSMP, a free response item, more teachers wrote 
about the healthy attitudes of their students than about anything else. 

Teachers felt that students' achievement was higher with CSMP than compared 
to previous years with other math programs. Fifty percent of the teachers 
thought achievement was "far more" and another 37% thought it "a little more" 
than the previous program, while only 4% thought it either "a little less" or 
"far less." Two and a half times as many responses were listed for "areas in 
which studenf^ have accomplished more" than for "areas in which students have 
accomplished less." Half of the responses listed under "more accomplished" 
dealt with basic skills, especially computation, and over a quarter had to do 
with general abilities such as problem solving, critical thinking, relationship, 
etc. These responses coincide fairly well with ti^st data gathered during the 
past two years*** 



*For the complete report, see 2-C-l, Teacher Questionnaire Data 
**E valuation Report l-A-3: Final Summary Report Year 1 and 
Evaluation Report 2-B-l: Second Grade Test Data 
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There were signs that the program appeared to be being implemented fairly 
well. For example: , 

a) Most teachers thought that the spiral approach was more effective 
than the mastery approach. 
. b) Second-year teachers generally felt their classes went more 
smoothly this year. 

c) Most teachers said that preparation time for CSMP was about the 
same, or would be after a year's experience, as for other math 
programs . 

d) As one might expect, the content given the heaviest emphasis by 
teachers had to do with basic computational skills and the least 
emphasis was given to ideas such as probability and combinatorics 
topics which appear infrequently in CSMP and virtually never in 
other programs. 

However, there were indications of things happening which were not intended. 
For example. 

a) Three quarters of the teachers supplemented the program with Non-CSMP 
material such as commercial worksheets. In half of the cases this 
was done on a regular and frequent basis. 

b) Significant numbers of teachers (one-third of the first grade 
teachers) taught math for over one hour per day, rather longer 
than usual. 

c) Forty percent of the teachers received less than 10 hours of 
teacher training, or less than half the recommended time. 

d) Almost a quarter of the teachers rate the ease of managing CSMP 
materials as either ''poor" or ''unsatisfactory" and over half 
thought that a better bookkeeping system was needed for monitor- 
ing student progress. 

e) Many second and third grade teachers did not make sufficient 
progress during the year to complete even the minimum recommend- 
ed portion of the curriculum. It should be noted, however, that 
teachers with a year's experience in CSMP did tend to make much 
better progress than teachers new to CSMP. 

In the authors' opinion, the two most serious criticisms had to do with the 
(possibly related) issues of the efficacy of the Minicomputer and the appropriate- 
ness of the program for low ability students. In a free-response item dealing 
with the Minicomputer, 37% of the first and second grade teachers did not think 
it was a good device for teaching low ability students; about the same number 
thought it was a good device for them. Forty-one percent of the teachers thought 
CSMP less appropriate for low ability students than their previous math program; 
46% thought it more appropriate. 

Tlie responses to these two questions, reflected fairly well in the summary 
evaluations given by each teacher, were rather different than, say, the responses 
to the question regarding tlie management of materials. In the latter case the 
responses were also balanced but this was because many neutral and few extreme 
responses were given. For the two questions described above, there were many 
responses at either end of f:he scale. Teachers felt rather strongly about tlie 
issues and happened to be rather equally divided one way or the other. Without 
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considering the reasons for their opinions, and noting that equal numbers of 
teachers felt just the opposite, it is true that over a third of the teachers 
thought the Minicomputer in particular, and the program as a whole, were not 
particularly appropriate for low ability students* Most of these same teachers 
nevertheless were, on the whole, impressed by CSMP students' achievement and 
attitudes • 

Teacher Interviews: Second Graded 

The interviews were conducted with the. 17 local second grade teachers at 
the end of the 1974-75 school year. These were teachers of classes which be- 
gan using CSMP materials in first grade the previous- year. The students had 
then completed two years of the CSI^ program, * ' 

Thr purpose of the interviews was to probe teacher opinion concerning 
the second grade CSMP program in a somewhat informal, open-ended manner. It 
was felt that through this personalized approach, information might be gained 
that would not have been gotten in the more formal questionnaires and logs 
which are given to all the teachers using CSMP. 

The interview was based on a set of questions designed to elicit a range 
of teacher opinions about the program including general impressions, program 
comparisons, best and worst aspects, time in class, lesson sequence, management 
of materials, student attitudes, new studencs, slow students, teacher prepara- 
tion, parent reactions, first and second grade comparisons, and other comments. 
In some of the questions the teachers were asked to compare CSMP to other programs 
in terms of certain aspects. In other questions they were asked to make non- 
comparative qualitative judgements about CSMP, 

Usually the interviews took place while the students were involved in part 
of the end-of-year testing program. The interviews took from 20-30 minutes to 
administer and were usually conducted in private with only the interviewer and 
the teacher present. One interviewer conducted all the interviews. All the 
interviews were tape recorded. Transcripts of the recordings are included in 
the Appendix of Report 2-C-2, 

Using the transcript of the interviews, an attempt was made to summarize 

the responses across teachers. Because of the open-endedness of the questions 

(and the extent of the responses), this proved to be a somewhat difficult task. 
Nevertheless, a number of conclusions may be drawn, 

1, The teachers were favorably impressed with the overall program 
and many specific aspects of it. Many indicated their opinion 
of the program had become more favorable as the year progressed, 
although some teachers still expressed concern about the program 
meeting the needs of the low ability student, 

2, The teachers felt the overall CSMP prograr was superior to other 
programs which they had taught, especially in regard to content, 
student attitude, and learning. 



*For the complete report, see 2-C-2, Teacher Interviews: Second Grade, 
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3. The Minicomputer and the variety of higher level concepts were 
viewed as being the best aspects of the program while the 
difficulties encountered by the low ability student were 
mentioned as the worst aspect. 

4. The teachers felt CSMP does not involve appreciably more class 
time than other programs and the suggested times for the lessons 
in the teacher's guide were reasonable. 

5. Generally the teachers followed the teacher's guide but did make 
slight adjustments in the lesson sequencv^ and used supplementary 
materials especially for basic addition and subtraction facts. 

6. The teachers found the materials easy to manage and felt the 
workbooks, worksheets and class activities supplied enough 
information to decide which students needed individual help. 
Also about half of the teachers indicated a need for testing 
materials or suggestions for topic emphasis. 

7. All the teachers felt the students enjoyed CSMT more than other 
math programs with many teachers feeling the average and above 
average students liked the program the most and the low ability 
students liked the program the least. 

8. Many teachers felt the students who entered the program in the 
middle of the year or later experience ' the most difficulties, 
and more attention should be paid to problem of new stvidents 
in the teacher's guide. 

9. There was little consensus of opinion concerning the appropriateness 
of the program or the spir-il approach for the slower students. 

10. Parent reactions, though not extensive in most cases, were almost 
always positive. 

11. All teachers indicated some special training was necessary to teach 
the program. 



These conclusions leave one with an overwhelming impression that the teachers 
have a very high regard for the program. The only probldms mentioned with any 
consistency were the difficulties (experienced by the slower students, and the 
need for more emphasis on the basi^ addition and subtraction facts. 

Teacher Interviews: First Grade* 

The interviews were conducted with 13 local first grade teachers at the end 
of the school year. These teachers all taught first grade CSMP during the previous 
year, 1973-74; at which time they were the lead group of first grade teachers. 
Thus this was their second year in teaching the first grade CSMP curriculum. 



*For the complete report, see 2-C-3, Teacher Interviews: First Grade 
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Presumably any "novelty effect" would have worn Oj:f and teachers would be con- 
fident enough of their knowledge of the program to have adapted it in whatever 
way they wanted. 

The interview was based on a series of questions relating to various 
aspects of the program: including background information, change from last 
year, new students, low ability students. Minicomputer and arrow diagrains, 
student attitudes, the spiral approach and an overall evaluation. The 
complete set of questions was not necessarily asked of each teacher; in 
some cases the teacher's response to a previous question provided the desired 
information and in other cases the interviewer simply missed the question. 
Occasionally a teacher raised new issues which were pursued in follow-up 
questions. 

The interviews required from 20 to 30 minutes and were usudly conducted 
at lunch or recess or after school when the children were not present. All 
the interviews were tape recorded and transcripts of the recordings are given 
in the Appendix of Report 2-C-3. The summary that follows below is the 
author's own version of the salient points. 

Five teachers will not be teaching CSMP next year. This may have caused 
them tc respond or to teach the program in a different way than would be the 
case if they knew their students would be continuing into second grade in CSMP. 

There was generally what might be called a "return to normality". Teachers 
were neither wildly enthusiastic nor extremely negative in their appraisals of 
various aspects of the program. They seemed to have roughly the same number of 
students (fairly typical class sizes), with approximately the same ability level 
and covere:d abcut the same number of lessons, as in the previous year. The 
program went mere smoothly for teachers, there seemed to be fewer problems in 
managing the materials (or else teachers got used to existing problems) and 
the amount of preparation was reduced to nearer what teachers would normally 
spend. Teachers adapted the program in various ways; in dealing with low ability 
students, in omitting certain lessons, in supplementing with Non-CSMP worksheets, 
in working with new students and i.n repeating or lengthening the instruction time 
for certain lessons. All these things indicate that the teachers were getting 
used to the program and had indeed incorporated it into their teaching so that 
CSMP was no longer "the new math program** but was becoming, in a sense, just plain 
"math". 

It is v;orth noting, however, that many teachers were still spending more 
time in teaching CSMP than they would have spent with their previous math pr.^ram. 
Whether this is good or bad is problematical. Some teachers indicated the program 
was rich with many varied things to do and they and their students enjoyed math 
time; thus it was desirable to spend more time. Others noted that there were so 
many lessons which required extra time and review, that in order to complete a 
reasonable portion of the curriculum it was necessary for them to spend longer 
each day. 

Without doubt the teachers, with one exception, liked the program and liked 
teaching it. Many of the responses, both positive and negative, were somewhat 
tempered and every teacher pointed out some facets of the program they disliked. 
Clearly these teachers with two years experience with CSMP did not usually see 
the program as superlative in every way, but just as clearly they did like it. 
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The most positive aspect of CSMP was the enthusiasm that students had for 

the program. This was attributed to the wide variety of lessons and materials 

and to the fact that students did not have to spend long periods of time on ,the 
same topic or skill development. • 

The questions which drew the lon»;est responses and the most disagreement 
were those regarding the appropriateness of the program for low ability students 
and the value of the Minicomputer as a teaching device. These two questions were 
stroagly related of course, since there was fairly strong agreement that for 
average, and especially high ability students, the program as a whole and the 
Minicomputer in particular wnre vnry effective and the teachers themselves liked 
the Minicomputer. The students learned more and were presented with a greater 
challenge than would normally be the case. But teachers did not agree on the 
efficacy of the program for low ability students. Some thought the Minicomputer 
and arrow diagrams too abstract, particularly near the beginning of the year when 
students did not have a fully developed concept of numbers and all the teachers 
indicated that these studentie^id not get as much out of the program as they would 
have liked. Many noted th^t these students could not work on their own with many 
activities. One may take it that these teachers feel that CSMP does not adequate- 
ly meet the needs of low ability students. However, in the view of most, neitaei 
do other math programs. Teachers were divided on whether or not t! ese childrevi 
would be better off in a traditional program; it seemed to be a difficult decision 
either way. Similarly, whether or not these students got enough out of the 
Minicomputer to justify the time and effort was a difficult decision. Typically 
teachers seemed to feel that kids liked it, that slow children hiid many difficul- 
ties with it but that they got something worthwhile out of it. 

\ 

Certain individual interviews highlighted certain issues or raised new ones. 
For example: 



a) The need for a screening test at the beginning of the 
year to identify students ill prepared to begin CSMP. 

b) The difficulties in dealing with large numbers of 
students who transfer into the class during the 
school year without the CSMP background and language. 

c) The difficulty in teaching CSMP without assistance 
after a year of frequent help from aides, senior 
students, etc. 

d) A description of the basic thinking skills enhanced 
by CSMP. 

e) A well articulated criticism of various aspects of 
the program (the only interview that was on the whole 
negative towards CSMP) . 
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Review of Evaluation Questions 

At the outset of this long-term evaluation effort (called the Extended Pilot 
Trial (EPT) of the CSMP elementary school math program), a set of rather specific 
questions were posed (page 33, Report 1-A-l, Overvie w, Design and Ins trumentation) > 
At the time, these questions served the purpose of giving the reader a better idea 
of exactly what would be the foci of this sununative evaluation • 

Now, in this final section of the summary report for the second year of the 
EPT, it would seem appropriate to again raise those same questions. This may 
accomplish three purposes. (1) Check to see if the actual evaluation effort is 
being focused on the issues deemed important at the outset. (?) See if there 
are now answers (or partial answers) to the questions • (3) Examine the necessity 
for revisions of, or additions to, the initial questions* 

The irlrial questions fell into three categories of issues: the intrinsic 
merit of t ie :MP program, the practicality of its adoption by regular school 
systems, and the cognitive outcomes of the program on children studying it. 

Intrinsic Merit 



During the first two years of the EPT, there have been two expert, external 
reviews of the CSMP program. The first of rhese took place during Year 1 of the pilot 
study and constitutes Evaluation Report l-A-2. Five mathematics educators, 
selected from a list provided by the Mathematical Association of America, indepen- 
dently reviewed available CSMP curricular materials. The second Te\^ie^;' consisted of 
site visit of CSMP conducted by a three person team selected by the National 
Institute for Education • In the Appendix is given that section of the resulting 
report which deals with the CSMP materials. 

Regarding the soundness of the mathematical content, there has been univer- 
sally high praise in both reviews. Regarding the relevance of the program, review- 
ers differed greatly in their apparent definitions of relevance and one hesitates 
to make a general summary of the reactions. Nevertheless, the program has been 
generally commended for its innovations in the area of presenting the basics and 
for its vigorous inclusion of topics either not presented in elementary programs 
or presented very poorly or in a minor way. 

On the issue of overall soundness of the program, reviewers had mixed and 
tentative reactions. The reactions were mixed in that not all reviewers reacted 
positively to each of the major pedagogical techniques of the program. The 
reactions were tentative in that many of the reviewers exyjressed the pragmatic 
desire to withhold judgment on the innovative pedagogy until its effectiveness 
could be determined. However, the overall judgment on this issue was also 
positive. 

Practicality 

There have been two approaches to obtaining information as to the practi- 
cality of the adoption of the CSMP program by regular school systems. The first, 
has been l:o compile figures on the extent of use of the program; presumably, 
analyzed closely, these figures will at least point to gross problem areas in the 
dissemination process, if any such arise. The other approach has been to try to 
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identify the specific requirements for the adoptability of the program and take 
measures of to what extent the CSMP experience in the pilot trial sites has met 
those requirements. 

Since the is only two years old, the evaluation effort has so far con- • 
centrated on the second approach, leaving a careful analysis of usage figures 
until later in the EPl when a sufficient amount of such datra is available. 

Using the second approach, one can review the data collected by turning tcj 
each of the initial evaluation questions which address the issue of "Practicality." 

What is the cost, to the adopting school systems of buying materials initially 
and maintaining materials for continued use, especially in comparison with present 
materials costs for elementary school math instruction? During the fall of 1974, 
a rather extensive analysis was made comparing costs for CSMP materials (K~2) with 
those of 13 commercial publishers (see pp. 21-2A of Report l-A-3). was found 
that although the CSMP costs were not way out of line and actually loss costly 
than some of the coiranercial materials, they certainly were higher than average. 
Although these relatively high costs are a very real concern, there are many 
factors involved between the present state of the materials and the final product 
and it is difficult to estimate what the eventual price might be. As of yot, no 
data has been collected on the extent to which cost has figured into the decisions 
of school systeias to (a) try oat the program (b) expand or contract its use, or 
(c) discontinue using CSMP materials. Clearly such data should be collected, 
where possible. 

What are the personnel requirements for a school system in using the program? 
These requirements are three-fold: a local coordinator, teacher training, and 
the possible availability of teachers with more specialized training in grades 3-6. 
Although a local coordinator is required by CSMP in its contract with the school 
system and is probably quite essential to the smooth adoption of the program in a 
systen, it has not been necessary for school districts to hire a special coordinator 
just for CSMP. The role of coordinator has perhaps best been played by district 
supervisory personnel, though classroom teachers, principals, and local university 
professors (people with a very wide variety of math backgrounds, power within the 
school systems, etc.) have been at times successful also. This is not to say that 
anyone can play the coordinator role successfully, but rather that individuals with 
Che formal qualifications required would seem to be present in most systems. 

As for teacher training requirements (up to one week, depending on grade 
level), these do not appear to be excessive and there seem to be enough success- 
ful variations on their fulfillment, that most any smalle r school system can 
handle it. Larger systems seem to have more difficulty fulfilling this require- 
ment. Finally, as to the question of the necessity of available teachers with 
more specialized training for grades 3 to 6, there is really no evidence at all: 
either as to the eventual necessity for such teachers or for the capability of 
school systems to avail themselves of such teachers. 

Although further elaboration on this question of personnel requirements is 
available (Report l-A-3, pp. 2,3), suffice it to summarize here that, for the 
CSMP program as it exists through Year 2 of the EPT, the personnePrequirements 
do not seem to be out of line with what most (but not all) school systems can 
provide. 
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How successful are teachers in coping with the program and implementing it 
in a manner reasonably faithful to the intentions of CSMP? Here and for Lhe 
next two questions the reader may wish to consult rather detailed answers to this 
compound question in the reports on teacher logs, questionnaires, interviews and 
observations (those numbered n-C-n) or tne summaries of same in Report l-A-3 and 
the previous section of this r?port. Very briefly, the vast majority of the 
teachers who have taught CSMP seem to be able to cope with it and do so in a 
manner reasonably faithful to the original intentions. 

It is important to note that the usual pattern of implementation thus far 
is with teachers who, by and large volunteer to teach the program. Whether teachers 
who are more or less forced to teach CSMP (as could be the case with a system-wide 
adoption) will cope as well and implement the program as faithfully remains to be 
seen. Clearly this question is one which can't be given an adequate answer until 
there has been more experience with school systems who have adopted the program 
system-wide . 

Do users like the program in comparison to other math programs they have used? 
If one defines "users" to mean "teachers", a great deal of evidence has been 
gathered, and the answer is clearly positive. Again it should be pointed out that 
relatively few teachers have taught CSMP who were completely uninvolved with the 
decision to use it. And again, this question must be left open until there has 
been more experience with system-wide adoption. 

If one defines "users" to mean "students", a great deal of secondary evidence 
has been gathered. Teachers almost universally report that the vast majority of 
their students really like the program. Since students at this age are almost 
never included in the decision to use a particular curriculum, one can assume 
that these teacher reports are on non-voluntary students. Rather limited attempts 
to assess student attitudes more directly have not shed much light on this issue. 

Can students transfer into and out of the program at any point in the curricu- 
lum without creating serious difficulties for the student or his new teacher? The 
question of students transferring into CSMP has been investigated rather thoroughly 
both in the evidence gathered from the teachers (the n-C-n reports) and in the 
analysis of student outcomes, (see 2-B-l, pp. 56,57). It is clear that transferring 
into the CSMP program can cause problems, but these seem generally not to be of 
great concern to teachers or school systems. 

No direct evidence is available on students transferring out of the CSMP 
program, but since the scores of CSMP students on basic skills and concepts are, 
if : >ything, higher than Non--CSMP students, one would not expect such students 
to h<-.ve difficult' with more traditional math programs. 

It may be that the problem of students transferring into CSMP will become 
more serious as students have a greater backlog of catching up to do. Or it may 
be less serious because students are older and can more quickly assimilate the new 
information. Hence this question needs much more time to be investigated. It 
should be pointed out here that we have been speaking of individual transferring 
students. There also exists-the situation in which an entire class eaters the 
program at other than first grade. In the future it should be possible to give 
evidence on this question, whether the transfers be individuals or class groups. 
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The above questions speak to the issue of whether or not (some number of) 
"typical" classrooms in a school system cm use the CSMP program. These 
questions may be thought of as parts of a larger question. 

Will it be possible for a school system to adopt the program syst em-wide? 
As lamentable as the idea may be to those who see alternative approaches to 
education as an answer to some of the problems faced by school systeins, many 
school systems take the view that curricular programs must eventually be either 
adopted district^wide or discontinued. 

What this implies, in terms of all questions raised in this category 
of "Practicality," is that eventually a school system must be able to give a 
non-negative response to each of them. That is to say, (a) the total regular 
budget for materials must be adequate to meet the costs of adopting the program 
system-wide. (b) The personnel requirements for system-wide adoption must not 
be outside the limitations of the regular staff cf the system. (c) All the 
teachers must be able to cope with a faithful implementation of the program, not 
just Uiose who would like to try it. (d) All the users (students and particularly 
teachers) must at least not totally dislike the program, not just those who more 
or less volunteered. In addition one might add that (e) the students in the 
district (as a whole) must be able to cope with the program and produce achieve- 
ment scores at least as high as expected from past experience. Obviously the 
transferring student question is not heightened (but rather, somewhat lessened) 
by the adoption of the program system-wide. 

The data summarized above has concentrated on the specific prerequisites 
deemed necessary for adoption. The other approach (that of analyzing the usage 
statistics) has been pursued only so far as to compile gross figures on the 
usage of CSMP materials as shown in the graph below. 
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Graph 1 
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Within a year or so, it should be possible to begin analyzing these gross 
figures more closely. For example, are the increases in CSMP usage the result 
of both continued and increased usage on the part of the systems which adopted 
the program initially and the addition of other adopting systems? Or are the 
increases due to the large number of additional adopting systems which more 
than offsets the discontinued usage in some systems which adopted CSMP earlier? 

As the EPT moves into its third and fourth years, it should be possible 
to gather much better evidence, whichever of the two approaches is used. How- 
ever, perhaps a much more fruitful approach, from this point on, would be one 
vfhich uses and dovetails the evidence gathered separately under the two previous 
approaches. Specifically, by paying close attention to which school systems are 
(and are not) continuing with the CSMP program, we should be able to gather much 
more meaningful evidence on which minimal requirements for adoption are being met, 
which are not, and under what circumstances this evidence is occurring. 

Cognitive Outcomes for Students 

As pointed out above, one aspect of this ^'Outcomes" issue would place it 
under the adoptability side of the issue of "Practicality." Nevertheless, since 
the whole point of developing a new and different math curriculum is to enable 
students to acquire the skills in arithmetic and the beginnj.ngs of understanding 
mathematics, the issue of cognitive outcomes for students doserves a category of 
its own. As a result, a major part of the evaluation effort has been devoted to 
gathering the best possible evidence to shed light on the issue. The reader's 
attention is called to the eight reports written on this issue so far in this 
series (numbered n-B-n) . Herein, each of the three questions raised initially 
on this issue shall be reviewed. 

Do students learn the basic concepts and skills, particularly computational 
skills, generally expected of students in elementary school? To gather evidence 
on this question, traditional tests of math achievement have been used. During 
the first two years of the EPT, the testing has involved almost exclusively 
those sites located in the St. Louis area. Therefore any statements in reference 
to data collected in answer to this question must be qualified by the fact that 
the evidence comes from not necessarily "typical" classrooms. 

The evidence gathered so far has given a resounding, and almost universally 
positive, response to the question. On the average, CSMP students do better on 
traditional achievement tests than Non-CSMP students. In addition, similar 
results were obtained for students in each quartile of academic ability (as 
obtained with standard tests of same). The results in favor of CSMP students 
held true in first grade and second grade, and in urban, suburban and rural 
school districts. 

To these generalisations, the only qualifiers which should be added (to the 
one in reference to "local" sites, above) are in regard to teacher quality, 
curriculum purity and differential time spent on math in CSMP classrooms. 

(a) It is possible that, since teachers of CSMP have been volunteers (by and 
large), the positive test results are due to their ^enthusiasm. Although one 
might counter by saying that enthusiastic teachers (stimulated by CSMP) are 
precisely what one might desire, the question still remains as to whether 
teaching CSMP will enthuse non-volunteer teachers to an equivalent extent. 

(b) A similar set of arguments and counter arguments can be raised on the 
issue which arises from the fact that CSMP teachers spend more time teaching 
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math than do Ncn-CSMP teachers. (c) As for the .fact that not all CSMP teachers 
use CSMP materials exclusively, again the same arguments can be raised. However, 
in this case, it is not so clear to what extent CSMP materials are supplemented 
by other math materials, nor what effect varying amounts of supplementation have 
on these achievement test results. 

Hence we see that, to the not inconsiderable extent to which data has been 
collected In regard to this question, the answer is "y^s." As always with re- 
search, the answer must be qualified, in this case by the above qualifiers.^ 
Thus, it is evident that additional research must be done, even on the primary 
grades, iu addition to that involving grades 3-6, in years to come. 

Do students learn the specific skills and concepts of the CSMP curriculum? 
Evidence in regard to this question has been gathered on a wide variety of 
students (just as in the one above). However, since the CSMP curriculum is 
unique in many respects and since it is really unfair to ask school officials 
their permission to test Non-CSMP students in areas that they are unfamiliar 
with, no comparative data has been collected. 

Nevertheless, considerable amounts of direct and indirect evidence have 
been gathered on CSMP students to provide a qualified answer to the question. 
CSMP students have demonstrated some success with a variety of items, involving 
the acquisition of the skills and concepts of the CSMP curriculum. Naturally, 
not all students demonstrate complete mastery (nor is it required in the program). 
As a result, it is very difficult to interpret the results obtained. As for 
indirect evidence, teachers report that students do acquire the skills and concepts 
of the CSMP curriculum, with the exception of some teachers who report not unexpect- 
ed difficulties with less able students. 

Are CSMP students, particularly after two or three years in the program, 
better able to deal with certain kinds of mathematical situations than are 
students who have not studied CSMP? So far, students have received, at most, two 
years of CSMP instruction, and hence it may be too soon to give an adequate answer 
to this question. However, attempts have been made to gather evidence in this 
most difficult area. 

Several test scales have been devised, for both group and individual admin- 
istration, which involve relatively novel mathematical situations for both CSMP 
and Non-CSMP students. While these have been developed in the absence of a 
strong overall theoretical framework, the results have indicated that CSMP 
students are indeed better able to deal with certain mathematical situations, 
there being no significant differences on the others. 

For the three questions dealing with the issue of "Outcomes," the answers 
thus far are largely positive, but limited primarily by the facts that the EPT 
is only two years old and that most of the data comes from "local" schools. 

From the foregoing treatment of the evaluation effort to date, the reader 
can easily surmise the kind of emphasis the evaluation effort will be having 
in the near future. Clearly, both kinds of evidence gathered on the "Practicality" 
issue will be looked at more closely and in confunction with each other, particu- 
larly as it relates to system-wide adoptions. Also, the data gathered on the 
"Outcome" issue should henceforth come from not only "local" sites but also from 
sites in the "outer ring", where the implementation effort approximates more close- 
ly the situation that would most likely prevail as, the program is more extensively 
Q adopted nation-wide. 
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Appendix 



Excerpt from the Report of a Site Visit to the 
Comprehensive School Mathematics Program 

May 19 to May 22, 1975 
by 

Jack Churchill, Shirley Frye, Gail Young 



"A, The CSMP Program 

We believe that CSMP has succeeded in opening up some of the riches of 
mathematics to elementary students in a way that can be taught with success 
by teachers given the minimal inservice programs developed by the project, 
CSMP's stated view of mathematics, their objectives and goals in the teaching 
of it, and their spiral curriculum approach impress us as sound and well 
exemplified in their materials • 

Indeed, wp. find the materials often richer than is exploited by particular 
lessons. This is as it should be, since it allows creative teachers and students 
to explore these riches as far as their interests and abilities allow. In short, 
we find CSMP delivers well on what it promises to do, and we heartily approve of 
the prom^.se. 

It is quite possible that the long period of developmental work by CSMP, 
including trials of various approaches and instructional modes, was a necessary 
preparation to the successful development of the project's current course 
materials. In other words, curriculum projects of the pase may have fallen short 
simply by not persevering long enough to develop materials that were at the same 
time sufficiently rich in mathematics and successfully usable by teachers and 
students. In this we believe CSMP has certainly succeeded to an extent that 
deserves support for the development of its A-5-6 curriculum. 

• A characteristic of the CSMP program is its emphasis on mathematical ideas, 
treated in a unified fashion. The matheiucicician can recognize in the content of- 
the K-3 program an imposing list of concepts: set intersection and union, function, 
inverse function, parallel displacement, Ven.n diagrams, functional composition, 
bases other than 10, algorithms, permutations, and the number line, to name just 
a fewk — It is 20c:h century mathematics; the old elementary curriculum was 15th 
century mathematics. 
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But even to list these terms is to give a misleading impression of the program. 
Neither the teacher nor the children will be aware that they are studying such 
'advanced* or * modem' topics. 

Other curriculum projects introducing such mathematical concepts have, by 
and large, tried to give the children a conscious, verbal, awareness of the 
mathematical formalisms in which such concepts are described. Mathematicians 
and mathematical educators have in their professional training all passed through 
the experience ot realizing that 5 and -5 are each the additive inverses of the 
other in the ring of integers, and found that at that stage formalism was illumi- 
nating, and many have attempted to recreate that experience in the elementary 
school curriculum. These attempcs have not been particularly successful (though 
an improvement on the older curricula), partly because they are not natural to 
the child, but also because such an approach makes greater demands on the mathe- 
matical background of the teacher than is realistic in this country. 

The CSMP program is based on much sounder pedagogical principles. One is 
reminded of M. Jourdain in Le Bourgeois Gentilhomme and his discovery that he 
had been speaking prose all his life. The CSMP children will discover with the 
same surprise that they have been studying all of these impressive terms all 
their school days. 

Wnat makes all this feasible is the heavy use by CSMP on non-verbal languages. 
There are three principal ones of these. The language of strings replaces the 
formalism of set theory; the language of arroi^s replaces the formalism cf functions; 
and the 'Minicomputer* replaces the formalism of algorithm and algebra. 

In the classroom, all of these pedagogical devices appear simple and natural. 
Neither the children nor the teacher have any difficulty in grasping them. 

The Minicomputer deserves some special comment. The name suggests some 
electronic device. An abacus will be a much better analogy. In essence, it 
consists of a number of squares, each subdivided into four squares,* a number 
of markers such as checkers, and several simple rules for manipulation. It is 
a computer in the same sense that the abacus is; foj.lowing the rules enables 
the child to carry out arithmetical computations well beyond what one can 
expect the child to do with the traditional methods at a given age. i ut it is 
primarily pedagogical, a tool to be dropped as soon as the child is able to 
spare it. 

We will not attempt to describe the device here, but refer to the CSMP 
material. One can discuss it in quite sophisticated terras, as a hybrid binary- 
decimal computer. It has been criticized from that standpoint, as forcing the 
child to use binary arithmetic in doing ordinary arithmetic. Such criticisms 
m!ist have been made in absence of classroom observation. The child is not doing 
anything more sophisticated than saying that one plus one equals two, two plus 
two equals four, four plus four equals eight. It is a litt].e easier for him to 
represent these additions on the computer than it is to represent three plus three 
equals six, but that he is, in any sense, using two different bases will never 
occur to him, nor to his teacher, nor from our observations, does the child have 
any problems dealing with the computer. On the contrary, their facility with 
mental arithmetic and arithmetic in standard form was really impressive, well 
beyond their grade levels in the ordinary curriculum. 
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In any case, in a world of real computers, early familiarity with the basic 
ideas of computers seems to us well worth striving for, even in this very simple 
form. We were told, as a cute story, of a second grader who had been playing 
with one of the little electronic hand calculators now so conunon. He was asked 
what he thought was inside it, and he replied, 'A little man with a Minicomputer,' 
On reflection, the answer seems to be about as good as one could ever expect from 
a second grader. It shows a clear understanding of the possibilities of mechani- 
zing computation, and if one replaces the little man by an electrical source of 
power, and the Minicomputer by the various binary circuits, it is not too far 
from a simplified technical explanation. 

We can summarize what appears to us to be unique features of the program, 

A, Content: 

Includes the development of number systems and their operations and 
introduction of geometry, measurement and probability in all grades, 
— is based on the unifying ideas of set, function, relations which 

are creatively incorporated without complicated symbolism, 
— leads students to problem-solving techniques and applications of 

skills. 

— does teach the student the computation skills that are usually 
included in an elementary mathematics program, but through non- 
standard methods. 

— the transfer of concepts from 1-digit numbers to multi-digit 
numbers occurs in a casual way that seems to be very natural to 
the students. 

E, Pedagogy: 

— Minicomputer - both a cotonting and computing model. It reinforces 
quantifying as well as place value concepts. As a student tool, it 
is readily manipulated and enables the student to visualize an 
exchange of '2 for 1' much more easily than(a)*10 for 1' exchange. 
As a demonstration model, it is extremely usable by the teacher 
for large or small group instruction. 

— the languages of strings and arrows generate math concepts and 
ideas and develop understanding through child-related discussion. 

— story books - 'bring children into contact with mathematics at 
levels until now unexplored in education.' 

C. Teacher's Guides: 

— variety of presentation methods 
— management suggestions 

— flexibility of use with levels ^''f'^) of material in strands 

— content clearly explained 

— implementation at various stages suggested for teacher 

— spiral approach permits teacher to move through strands without 

expecting a 'mastery level' for each student 
— good suggestions for student activities other than the workbook pages. 

Because of the outstanding organization aiid dialogue in the guides, 
teacher in-service training required is very brief.'* 



